Introduction to the special issue on codes on graphs and iterative algorithms

I N the 50 years since Shannon determined the capacity of ergodic channels, the construction of capacity-approaching coding schemes has been the supreme goal of coding research. Finally today, we know of practical codes and decoding algorithms that can closely approach the channel capacity of some classical memoryless channels. It is a remarkable fact motivating this special issue that all known practical, capacity-approaching coding schemes are now understood to be codes defined on graphs, together with the associated iterative decoding algorithms. Considering the intensity of the efforts to achieve Shannon’s limit in the past few decades, it is ironic that the main ideas pertaining to codes on graphs and to sum-product decoding were, in essence, invented 40 years ago by Gallager but were subsequently neglected by the coding community. It is some consolation to recognize that Gallager’s codes were ahead of their time—given the limited processing capabilities of the time, Gallager’s codes were simply considered impractical. Even so, a small number of researchers continued to study codes on graphs. Zyablov and Pinsker (1975) and Margulis (1982) studied Gallager’s low-density parity-check codes. In 1981, Tanner wrote a landmark paper that formally introduced the graphical model notation for describing codes, proved the optimality of the sum-product algorithm in cycle-free graphs, and founded the topic of algebraic methods for constructing graphs suitable for sum-product decoding. Recent excitement about codes on graphs and sum-product decoding was ignited in the mid-1990s by the excellent performance exhibited by the turbo codes of Berrou and Glavieux, MacKay and Neal’s near-capacity results on Gallager codes, and the linear-complexity expander graph codes of Sipser and Spielman. At the same time, other researchers recognized a unifying theme in the iterative decoding algorithms and in the mid-1990s papers showing the connections between iterative decoding of codes on graphs and algorithms in the artificial intelligence and systems theory communities were published by Wiberg, Loeliger, and Koetter (1995), Frey and Kschischang (1996), McEliece (1997), McEliece, MacKay, and Cheng (1998), Kschischang and Frey (1998), and Aji and McEliece (1998). By now there exists a number of classes of codes on graphs that can approach the Shannon limit quite closely with moderate complexity, or extremely closely with high but not infeasible complexity. Indeed, ideas presented in this special issue have allowed us to approach the Shannon limit to within hundredths of a decibel.